Towards Semantic Fast-Forward and Stabilized Egocentric Videos

The emergence of low-cost personal mobiles devices and wearable cameras and the increasing storage capacity of video-sharing websites have pushed forward a growing interest towards first-person videos. Since most of the recorded videos compose long-running streams with unedited content, they are tedious and unpleasant to watch. The fast-forward state-of-the-art methods are facing challenges of balancing the smoothness of the video and the emphasis in the relevant frames given a speed-up rate. In this work, we present a methodology capable of summarizing and stabilizing egocentric videos by extracting the semantic information from the frames. This paper also describes a dataset collection with several semantically labeled videos and introduces a new smoothness evaluation metric for egocentric videos that is used to test our method.Comment: Accepted for publication and presented in the First International Workshop on Egocentric Perception, Interaction and Computing at European Conference on Computer Vision (EPIC@ECCV) 201

Antimicrobial resistance in Mycobacterium tuberculosis : mechanistic and evolutionary perspectives

Antibiotic-resistant Mycobacterium tuberculosis strains are threatening progress in containing the global tuberculosis epidemic. Mycobacterium tuberculosis is intrinsically resistant to many antibiotics, limiting the number of compounds available for treatment. This intrinsic resistance is due to a number of mechanisms including a thick, waxy, hydrophobic cell envelope and the presence of drug degrading and modifying enzymes. Resistance to the drugs which are active against M. tuberculosis is, in the absence of horizontally transferred resistance determinants, conferred by chromosomal mutations. These chromosomal mutations may confer drug resistance via modification or overexpression of the drug target, as well as by prevention of prodrug activation. Drug resistance mutations may have pleiotropic effects leading to a reduction in the bacterium's fitness, quantifiable e.g. by a reduction in the in vitro growth rate. Secondary so-called compensatory mutations, not involved in conferring resistance, can ameliorate the fitness cost by interacting epistatically with the resistance mutation. Although the genetic diversity of M. tuberculosis is low compared to other pathogenic bacteria, the strain genetic background has been demonstrated to influence multiple aspects in the evolution of drug resistance. The rate of resistance evolution and the fitness costs of drug resistance mutations may vary as a function of the genetic background

Improving Sequential Determinantal Point Processes for Supervised Video Summarization

It is now much easier than ever before to produce videos. While the ubiquitous video data is a great source for information discovery and extraction, the computational challenges are unparalleled. Automatically summarizing the videos has become a substantial need for browsing, searching, and indexing visual content. This paper is in the vein of supervised video summarization using sequential determinantal point process (SeqDPP), which models diversity by a probabilistic distribution. We improve this model in two folds. In terms of learning, we propose a large-margin algorithm to address the exposure bias problem in SeqDPP. In terms of modeling, we design a new probabilistic distribution such that, when it is integrated into SeqDPP, the resulting model accepts user input about the expected length of the summary. Moreover, we also significantly extend a popular video summarization dataset by 1) more egocentric videos, 2) dense user annotations, and 3) a refined evaluation scheme. We conduct extensive experiments on this dataset (about 60 hours of videos in total) and compare our approach to several competitive baselines

Localizing the Common Action Among a Few Videos

This paper strives to localize the temporal extent of an action in a long untrimmed video. Where existing work leverages many examples with their start, their ending, and/or the class of the action during training time, we propose few-shot common action localization. The start and end of an action in a long untrimmed video is determined based on just a hand-full of trimmed video examples containing the same action, without knowing their common class label. To address this task, we introduce a new 3D convolutional network architecture able to align representations from the support videos with the relevant query video segments. The network contains: (\textit{i}) a mutual enhancement module to simultaneously complement the representation of the few trimmed support videos and the untrimmed query video; (\textit{ii}) a progressive alignment module that iteratively fuses the support videos into the query branch; and (\textit{iii}) a pairwise matching module to weigh the importance of different support videos. Evaluation of few-shot common action localization in untrimmed videos containing a single or multiple action instances demonstrates the effectiveness and general applicability of our proposal.Comment: ECCV 202

Unsupervised Video Summarization via Attention-Driven Adversarial Learning

This paper presents a new video summarization approach that integrates an attention mechanism to identify the signi cant parts of the video, and is trained unsupervisingly via generative adversarial learning. Starting from the SUM-GAN model, we rst develop an improved version of it (called SUM-GAN-sl) that has a signi cantly reduced number of learned parameters, performs incremental training of the model's components, and applies a stepwise label-based strategy for updating the adversarial part. Subsequently, we introduce an attention mechanism to SUM-GAN-sl in two ways: i) by integrating an attention layer within the variational auto-encoder (VAE) of the architecture (SUM-GAN-VAAE), and ii) by replacing the VAE with a deterministic attention auto-encoder (SUM-GAN-AAE). Experimental evaluation on two datasets (SumMe and TVSum) documents the contribution of the attention auto-encoder to faster and more stable training of the model, resulting in a signi cant performance improvement with respect to the original model and demonstrating the competitiveness of the proposed SUM-GAN-AAE against the state of the art

Video Summarization Using Deep Semantic Features

Computer Vision - ACCV 2016: 13th Asian Conference on Computer Vision, Nov 20-24, 2016, Taipei, TaiwanThis paper presents a video summarization technique for an Internet video to provide a quick way to overview its content. This is a challenging problem because finding important or informative parts of the original video requires to understand its content. Furthermore the content of Internet videos is very diverse, ranging from home videos to documentaries, which makes video summarization much more tough as prior knowledge is almost not available. To tackle this problem, we propose to use deep video features that can encode various levels of content semantics, including objects, actions, and scenes, improving the efficiency of standard video summarization techniques. For this, we design a deep neural network that maps videos as well as descriptions to a common semantic space and jointly trained it with associated pairs of videos and descriptions. To generate a video summary, we extract the deep features from each segment of the original video and apply a clustering-based summarization technique to them. We evaluate our video summaries using the SumMe dataset as well as baseline approaches. The results demonstrated the advantages of incorporating our deep semantic features in a video summarization technique

Summarizing Videos with Attention

In this work we propose a novel method for supervised, keyshots based video summarization by applying a conceptually simple and computationally efficient soft, self-attention mechanism. Current state of the art methods leverage bi-directional recurrent networks such as BiLSTM combined with attention. These networks are complex to implement and computationally demanding compared to fully connected networks. To that end we propose a simple, self-attention based network for video summarization which performs the entire sequence to sequence transformation in a single feed forward pass and single backward pass during training. Our method sets a new state of the art results on two benchmarks TvSum and SumMe, commonly used in this domain.Comment: Presented at ACCV2018 AIU2018 worksho

How Local is the Local Diversity? Reinforcing Sequential Determinantal Point Processes with Dynamic Ground Sets for Supervised Video Summarization

The large volume of video content and high viewing frequency demand automatic video summarization algorithms, of which a key property is the capability of modeling diversity. If videos are lengthy like hours-long egocentric videos, it is necessary to track the temporal structures of the videos and enforce local diversity. The local diversity refers to that the shots selected from a short time duration are diverse but visually similar shots are allowed to co-exist in the summary if they appear far apart in the video. In this paper, we propose a novel probabilistic model, built upon SeqDPP, to dynamically control the time span of a video segment upon which the local diversity is imposed. In particular, we enable SeqDPP to learn to automatically infer how local the local diversity is supposed to be from the input video. The resulting model is extremely involved to train by the hallmark maximum likelihood estimation (MLE), which further suffers from the exposure bias and non-differentiable evaluation metrics. To tackle these problems, we instead devise a reinforcement learning algorithm for training the proposed model. Extensive experiments verify the advantages of our model and the new learning algorithm over MLE-based methods

Population genomics of Francisella tularensis subsp. holarctica and its implication on the eco-epidemiology of tularemia in Switzerland

Whole genome sequencing (WGS) methods provide new possibilities in the field of molecular epidemiology. This is particularly true for monomorphic organisms where the discriminatory power of traditional methods (e.g., restriction enzyme length polymorphism typing, multi locus sequence typing etc.) is inadequate to elucidate complex disease transmission patterns, as well as resolving the phylogeny at high resolution on a micro-geographic scale. In this study, we present insights into the population structure of; Francisella tularensis; subsp.; holarctica; , the causative agent of tularemia in Switzerland. A total of 59; Fth; isolates were obtained from castor bean ticks (; Ixodes ricinus); , animals and humans and a high resolution phylogeny was inferred using WGS methods. The majority of the; Fth; population in Switzerland belongs to the west European B.11 clade and shows an extraordinary genetic diversity underlining the old evolutionary history of the pathogen in the alpine region. Moreover, a new B.11 subclade was identified which was not described so far. The combined analysis of the epidemiological data of human tularemia cases with the whole genome sequences of the 59 isolates provide evidence that ticks play a pivotal role in transmitting; Fth; to humans and other vertebrates in Switzerland. This is further underlined by the correlation of disease risk estimates with climatic and ecological factors influencing the survival of ticks

Local adaptation in populations of Mycobacterium tuberculosis endemic to the Indian Ocean Rim

Background: Lineage 1 (L1) and 3 (L3) are two lineages of the Mycobacterium tuberculosis complex (MTBC) causing tuberculosis (TB) in humans. L1 and L3 are prevalent around the rim of the Indian Ocean, the region that accounts for most of the world's new TB cases. Despite their relevance for this region, L1 and L3 remain understudied. Methods: We analyzed 2,938 L1 and 2,030 L3 whole genome sequences originating from 69 countries. We reconstructed the evolutionary history of these two lineages and identified genes under positive selection. Results: We found a strongly asymmetric pattern of migration from South Asia toward neighboring regions, highlighting the historical role of South Asia in the dispersion of L1 and L3. Moreover, we found that several genes were under positive selection, including genes involved in virulence and resistance to antibiotics . For L1 we identified signatures of local adaptation at the esxH locus, a gene coding for a secreted effector that targets the human endosomal sorting complex, and is included in several vaccine candidates. Conclusions: Our study highlights the importance of genetic diversity in the MTBC, and sheds new light on two of the most important MTBC lineages affecting humans